Indonesian Journal of Electrical Engineering and Computer Science 
Vol. 11, No. 2, August 2018, pp. 549~557 
ISSN: 2502-4752, DOI: 10.1159 l/ijeecs.v11.12.pp549-557 O 549 


Evaluation of Color Models for Palm Oil Fresh Fruit Bunch 


Ripeness Classification 


Nurbaity Sabri’, Zaidah Ibrahim’, Dino Isa” 


‘Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Campus Melaka, 77300 Melaka 
"Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Campus Shah Alam, 40450 Selangor 
*Faculty of Engineering, University of Nottingham Malaysia Campus, Jalan Broga, 43500 Semenyih, Selangor 


Article Info 
Article history: 


Received Feb 12, 2018 
Revised Apr 17, 2018 
Accepted Apr 21, 2018 


Keywords: 


Fresh Fruit Bunch (FFB) 
Color models 
Color moment 


ABSTRACT 


This paper investigates the application of eight color models for automatic 
palm oil Fresh Fruit Bunch (FFB) ripeness classification with multi-class 
Support Vector Machine (SVM). Ripeness classification is important during 
harvesting to ensure that they are harvested during the correct ripe stage for 
optimum oil production. Since color is a_ significant indicator for 
agriculturists to determine the ripeness of FFB, it is critical to determine the 
right color model. Eight color models have been investigated namely, HSV, 
111213, LAB, XYZ, YCbCr, YIQ, YUV and RGB. Color moments were 
extracted from each of these color models for the classification of four stages 
of FFB ripeness that are unripe, under-ripe, ripe and over-ripe. A database of 
five hundred images of palm oil FFB has been constructed and experiments 


SVM showed that YCbCr and YUV outperform the other color models. 
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1, INTRODUCTION 

The quality of palm oil Fresh Fruit Bunch (FFB) is important due to high market demands on palm 
oil products such as consumption, cosmetic industry and many more. One of the main challenges in fruit 
industry is to identify the right fruit for harvesting. In order to preserve and maintain the quality of palm oil 
product, it is crucial to harvest palm oil FFB at the right stage. The profit of palm oil industry relies on the 
post harvesting level [1]. Therefore, by harvesting palm oil FFB at the right phase of maturity, it ensures 
ideal quality and optimized the amount of oil production. In Malaysia, Elaies Guneensis is the common 
species of palm oil that is available and mostly planted. The ripeness of this species is determined by 
Malaysian Palm Oil Board [2]. Table 1 shows some sample images of the four levels of ripeness stages for 
palm oil FFB. By looking at Table 1, we can see that these different levels of ripeness can be easily 
determined through color but these color images are influenced by illumination. Thus, it is very challenging 
to identify the right color since different outdoor lighting condition can affect the color of the images. 

Recently, various studies in computer vision for fruit ripeness classification such as watermelon, 
bananas, and tomatoes have been explored [3]. Most of these studies use color as the parameter to determine 
the ripeness stage that involves the extraction of useful information concerning the spectral properties of the 
fruit surface. There are several color models that can be investigated in order to create a reliable approach to 
classify the fruits into different classes of ripeness [3]. Therefore, selecting an appropriate and accurate color 
model is crucial for designing and modelling automated machine vision system for palm oil FFB ripeness 
classification [4]. 

The most common color model is RGB where any color is described by its intensities of Red (R), 
Green (G) and Blue (B). RGB values are suitable to be used for constant lighting environment since they are 
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affected by changing light intensities [5]. RGB is a typical color model used in many fruit grading research 
such as peaches [6], mangosteen [7], tomato [8] and avocadoes [9], [10]. RGB color model has been applied 
in palm oil FFB grading in [11]-[13] but RGB 1s easily influenced by illumination. 

Another color model that is commonly used to classify fruit ripeness is the descriptive parameters 
Hue, Saturation and Value (HSV) where Hue describes the color; Saturation represents the purity of that 
color while Value is its brightness. HSV is less affected by illumination changes compared to RGB [14]. 
HSV is used in [8] to determine the ripeness of tomato by extracting the Hue value from FFB image. In palm 
oil research, this color model increases the palm oil ripeness classification accuracy by 20% compared to 
RGB color model [2]. 


Table 1. The ripeness stage of palm oil FFB by Malaysian Palm Oil Board [2] 


Image Stage Color characteristic 





Unripe Purplish black 

Under-ripe Reddish black 
Ripe Red 

Overripe Reddish orange 





International Commission on []lumination (Commission internationale de l’éclairage) (CIE) Lab 
color model is associated with display system such as television or color monitor to adapt with human color 
response characteristics [15]. This color model has been applied ripeness grading of apple and 
tomato [16], [17]. In palm oil grading, this color model has been used by [2], [11], [18], [19] but at the 
segmentation stage where this stage removes the noises from the FFB palm oil image. This color model 
produces a good segmentation result where it manages to remove the unwanted areas in the image. 

YCbCr color model commonly use in operation in digital video and image compression [21]. High 
accuracy has been achieved in classification of ripeness stage for watermelon [19] and orange [22] but 
moderate for tomato [20] ripeness classification for a least noise background environment. In orange ripeness 
recognition, lowest error recognition with the most robust algorithm for orange recognition has been 
recorded [22]. 

The [11213 color model used in apple detection where it successfully distinguish object form 
background [23]. In orange recognition, this color model shows a high recognition error compared to other 
color model [22]. This color model has also been used to recognize dates. However, it produce a moderate 
result of chrominace component value [24]. 

The YIQ color model is often used in segmentation operation [25], [26] where Y stores the value of 
luminance, I stores the value of hue while Q is for saturation. Besides segmentation, it has also been used for 
classification purpose such as iris authentication [27], [28]. A comparative study between HSV and YIQ 
color models for orange fruit grading has shown that YIQ color model is better than HSV [28]. Besides that, 
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YIQ color model has also shown high accuracy performance during night environment for litchi 
recognition [29]. 

XYZ model is based on human trigonometric retina and it has been applied on agriculture product 
classification such as in apple grading system [30] and sorting a citrus fruit [31]. However, this result varies 
due to the application of different classifiers. It has also been used to reduce the image size or processing 
time of RGB color model for grading an image of agricultural product indoor and outdoor environment [32]. 

YUV color model defines a color space in terms of one luma (Y) and two chrominance (UV) and it 
has less computational time compared to other RGB color model [33]. The color model is closer to human 
perception rather than the conventional RGB [34]. It has been used in agriculture product where using a 
single value in YUV is good enough to achieve high classification accuracy such as in oranges and tomato 
recognition [22], [35]. 

Since various results have been produced by different color models, an investigation among these 
reported eight (8) color models is investigated in this paper. The color models are HSV, I112I3, LAB, XYZ, 
YCbRc, YIQ, YUV and RGB. They will be utilized to classify the palm oil FBB ripeness. Researches have 
been conducted to classify various stages of ripeness for palm oil FBB. According to [2] most of the palm oil 
grading focuses on two, three or four stages which is unripe, reddish black as underripe, red as ripe, and 
reddish orange as overripe. Research by [18], [36], [37] focus on two stages which are ripe and unripe of FFB 
and achieved high accuracy results. [3], [13], [38], [39] recognize three stages of FFB ripeness which is 
unripe, ripe and overripe while [2], [40] work on four stages which is unripe, underripe, ripe and overripe. 

Artificial Neural Network (ANN) has been utilized to classify palm oil FFB into four ripeness 
stages [2]. However, ANN takes a long processing time due to over-fitting issue. Research on color model 
evaluation have been done for medical plant identifier using texture features with different classifiers namely 
Stochastic Gradient Descent (SGD), k-nearest neighbour (KNN), Support vector machine (SVM), Linear 
Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA) classifiers [41]. SVM produces 
the best identification result compared to the other classifiers. Thus, this research investigates the 
performance of the eight color models mentioned earlier for four stages of palm oil FFB ripeness 
classification with Support Vector Machine (SVM). 

This paper is organized as follows: starting with introduction to palm oil FFB and description of 
each color space used in this research. The next section discusses the various color models evaluated in this 
research followed by discussions on results analysis. The last section concludes this paper with future work. 


2. RESEARCH METHOD 
2.1. Transformation from RGB to XYZ 

The values of XYZ are computed using linear transformation from RGB color coordinates as shown 
in Equation (1). 


xX 0.607 0.174 0.201][R 
Y| = |0.299 0.587 0.114] |G 
Z 0.000 0.066 1.117] LB (1) 











2.2. Transformation from RGB to LAB 

Lab is known as device-independant color model where L represents the image lightness, *a 
represents red and green, the blue/yellow coordinates of color-opponent are represented with *b[35]This 
color model is acquired from XYZ and well known color model according to human color perception system 
due to comprehensive range of colors and linear representation[35]. Equations (2) to (4) show the calculation 
of LAB color space. 


f 
L* = 116 (0.299R + 0.587G + 0.1148) /3 — 16 (2) 
a = 500| (1.006(0.607R + 0.174G + 0.2018) /3 — (0.299R + 0.587G + 0.1148) /3| (3) 
b* = 200| (0.299R + 0.587G + 0.1148) /3 — 0.846(0.066G + 1.1178) /3| (4) 


2.3. Transformation from RGB to YUV 
YUV is known as luminance-chrominance space where it represents one component of luminance 
and two components of chrominance. The luminance component is represented by Y produced from XYZ 
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color model while luminance of U and V are produced from the combination of the Y component. 
Equation (5) shows the conversion of RGB to YUV color model. 


R 

4 

B (5) 
2.4. Transformation from RGB to HSV 

HSV that is derived from RGB color model presented by Hue, Saturation and Value is a 

quantification of human color vision. It is widely used and more suitable color space for human eyes. HSL is 
the alternative color model for HSV where it is represented by H-Hue, S-saturation and L-lightness. Hue (H) 
is produced by the base color or shadesin the form of degree or number. Saturation (S) is the ratio of white 
color or brightness of animage. The value (V) represents the lightness of the color where 0 is black and the 
lightness increases if it is move away from black [42], [26]. Equations (6) to (9) illustrate the transformation 
from RGB to HSV color spaces: 


—0.147 — 0.289 0.436 


Y 0.2989 0.5866 0.1145 
ul = 
V 0.615 —0.515 — 0.100 





*(2R-G-B) 

2 
SSS SS——=>_= (6) 
,/ (R—-G)2-—(R-G)(G-B) 
max(R,G,B)+min(R,G,B) 


a (7) 


H = across 


Sy {noe 
~ (max(R,G,B)+min(R,G,B) 


\ for L<0.5 (8) 


max(R,G,B)—min(R,G,B) 
S= | forL 20.5 9) 
2.5. Transformation from RGB to I1I2I3 
The Intensity-red, Intensity-green, Intensity-blue (111213) color model was obtained through the 
decorrelation of the RGB color components using the K—L transform by [43] as in Equation (10). This color 
model is produced from RGB manipulation. It is usedto highlight the image that contains light effects 
specifically for glittery object [44]. 


iH 


2.6. Transformation from RGB to YIQ 

The YIQ color model where Y represents a luma (brightness of an image used in black and white 
television) information, / stands for “in-phase” and the Q represents “quadrature” is generated from 
clockwise rotation of U and V from YUV color model [45]. YIQ color model is generally used in National 
Television System Committe (NSTC) television standard [26]. This color model is intended to take 
advantage of human color-response shown in Equation (11). 


Y 0.2990 0.5870 0.1140 |[R 
I} = |0.5957 — 0.2744 — 0.3213] |G 
Q 


WwW | 
WwW | 
Ww | 


111 
1 1 1% 
aes | (s 
1 1) 18 
2 2 


2 (10) 


0.2115 — 0.5226 0.3111 (11) 


2.7. Transformation from RGB to YCbCr 
The YCbCrcolor model consists of the Y the luminance component, and description of chrominance 
component represented byCb and Cr channels. This color model used in skin color detection due to division 


of luminance from chrominance [46] as shown in Equations (12) to (14). 


Y = 0.299R + 0.587G + 0.114B (12) 


Cb = (B — Y) * 0.564 + 128 (13) 
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a) Feature Extraction 

There are many techniques used in color features and among the popular features are color moment 
[12], [36], [32] and color histogram [2], [22]. A high number of accuracy has been achieved using color 
moment on palm oil FFB recognition [36]. Therefore, this research implements color moment for color 
model analysis. 

Color moment feature is popular for color detection application due to its ability to deal with 
lighting and illumination condition specifically for palm oil grading using RGB color image[47].The mean, 
standard deviation and skewnessfor each color channel is calculated is shown in Equations (11) until (14). 


MOMENT 1: Equation (15) finds the average color value of an image where N represents the size of an 


image in pixels while the Pij represents the pixel intensity according to color channel such as H/S/V, Y/I/Q 
and Y/U/V in rows and columns. 


j=l 
Ei = ys Pij 
i= y PU 
N 


MOMENT 2: Equation (16), ol , 1S also known as the square root of the variance of the color distribution or 


(15) 


Standard Deviation where the current pixel is deducted from the mean El computed in Equation (16). This 


value is then square root to produce a standard deviation, Ol. 





jel 
G Pu - B02) 
7 (16) 


MOMENT 3: Equation (17), ST. for Skewness is a measure of the degree of asymmetry in the distribution. 


The current pixel is deducted from the mean value, Ei, and square root of three to produce the skewness, S1, 
value from each color channel. 





j=1 
Si = > Pi — Fi)3) 
N 


(17) 


b) Support Vector Machine (SVM) 

SVM is used to discriminate multiple objects from each other by grouping the attribute of each 
according object with homogeneous characteristic. It works by separating classes into positive and negative 
classes by searching for the maximum margin around hyper plane [8]. This research focuses on SVM 
classification to classify between four stages of FFB using polynomial kernel function.The flow of process of 
this research is shown in Figure 2. It starts with image acquisition, pre-processing, color feature extraction 
and classification at the end this process. The result is the classification of the FFB ripeness grading stage 
that is Over-ripe, ripe, unripe or under-ripe. 


HSV 

I,lals 

LAB 

me Color Roe ~ 
iets —— weirs —— FF Classification - Unripe 
i - Under-ripe 
YuY 

RGB 





Figure 2. The color model of palm oil FFB ripeness classification process 
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c) Data collection 

About 500 palm oil FBB color images were captured by a palm oil expert from an estate located at 
Batu Pahat, Johor DarulTakzim, Malaysia, that consists of 87 images of over-ripe FFB, 126 images of ripe 
FFB, 109 images of under-ripe FFB and 147 images of unripe FFB. These images are manually cropped to 
obtain the region of interest or the FFB area and eliminate the non-FFB areas such as trees, leaves and other 
objects. These color images are then transformed into 8 types of color models which are HSV, I[11I2I3, LAB, 
XYZ, YCbCr, YIQ, YUV and RGB using the equations explained earlier. Table 2 illustrates some sample 
images as a result of the 8 color models. 


Table 2. Four palm oil FFB ripeness stages with eight color models 
Stages 


Under-ripe 


Model Over- ripe Unripe 


RGB 


LAB 


bk 


HSV 


XYZ 


YCbCr 


YIQ 


YUV 





3. RESULTS AND DISCUSSION 

This experiment is conducted using MATLAB R2016a for feature extraction and Weka for the 
analysis tool. The data is randomly divided to 80% training and 20% testing. Table 3 shows the ripeness 
classification rate for each color model based on color moment features where it is computed using 
Equation 14. 


ree Correctly identified image 
Identification rate = ——_W1!__— 
total test images (14) 
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Table 3. Result of palm oil ripeness classification rate for the 8 color models 


Color Model Ripeness Classification rate (%) 

HSV 87.2 
111213 85.1 
LAB 97.2 
XYZ 87.2 
YCbCr 98.9 
YIQ 85.6 
YUV 98.9 
RGB 68.6 


By observing the results in Table 3, it clearly shows that YCbCr and YUV produce similar ripeness 
classification rate that is higher than the other color models. It can be concluded that both color models is 
more robust to illumination compared to other color models. Besides that, it also shows that these color 
models are less affected with glossiness and silhouette surface of an image compared to other color models. 
Both of these color models have the same incorrect classification of the overripe images that have been 
incorrectly classified as under ripe. This may be due to the different number of training data for each stage of 
ripeness. 

RGB color model without transformation shows a slightly low result compared to the other color 
models. This is due to high correlation between ripe, unripe, over-ripe and under-ripe palm oil FFB images. 
Besides that, this experiment also shows that the color models with discriminating power such as HSV, LAB 
and RGB are not associated with the ripeness classification rate. 


4. CONCLUSION 

This paper focuses on the effect of applying different color model for palm oil FFB ripeness 
classification. Eight color models have been compared in this research which is HSV, I11I2I3, LAB, XYZ, 
YCbCr, YIQ, YUV and RGB. Four levels of palm oil ripeness has been analyzed which is ripe, unripe, over- 
ripe and under-ripe. Each color model has its own characteristic and determination power. YCbCr and YUV 
color models produce the highest ripeness classification for palm oil FFB. Development of mobile 
application for ripeness classification will be applied in the future. Besides that, more data will be collected 
and more comparative study on features extracted from these color models and classifiers will also be 
conducted in the future. 
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